App Crawler: Opening Mobile App Black Boxes

1 MOTIVATION A June 2011 report1 showed that on average, a U.S. smartphone user spent 9% more time using apps (on smartphone) than browsing the Internet (on PC or smartphone). This growth in app usage is fueled by an increasingly large number of diverse and useful mobile apps: there are more than a million mobile apps available today in major app markets and around 15,000 apps are released per week (compared to about 100 movies and about 250 books released per week worldwide).2 Many of these apps such as recipe apps, news apps, deals apps, utility apps, etc. contain useful information. Unfortunately the information is hidden behind so-called “wall-gardens” that cannot be easily tapped into like other information on the Web. For example, the information inside apps is not crawlable by Web crawlers, and hence users cannot search it in existing search engines. Thus, users are relying more and more on in-app search, instead of using horizontal search engines such as Bing or Google. This is evidenced by a 2010 study 3 on iPhone users that reveals that 75% of overall mobile query volume are made inside apps. Similarly, ad systems do not have visibility into what app page contents a user is interacting with, and how, and hence they cannot show contextually relevant ads. It is strategically important for Microsoft to find ways to get visibility inside smartphone apps and to tap into this information source. In the AppCrawler project, we take a first step in addressing several challenges in achieving this goal.